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This  volume  contains  144  papers  summarizing  the  talks  and  posters  presented  at  the  fifth 
NIPS  conference  (short  for  “Neural  Information  Processing  Systems — Natural  and  Syn- 
thcric"),  held  in  Denver,  Colorado,  from  2-5  December  1991.  Since  iu  inception  in  1987. 
the  NIPS  conference  has  attraaed  researchers  from  many  disciplines  who  are  applying  their 
expenise  to  problems  in  the  field  of  neural  networks.  The  conference  and  the  following 
two-day  workshop  have  become  a  forum  for  presenting  the  latest  research  results  and  for 
leading  researchers  to  gather  and  exchange  ideas. 

The  1991  conference  maintained  the  high  level  of  exdtemeni  of  its  predecessors. 
Important  new  theoretical  results  were  presented  concerning  the  capability  and  generaliza¬ 
tion  performance  of  networks.  Of  particular  interest  are  papers  included  in  this  volume  by 
Vapnik,  MacKay,  Haussler,  and  others,  which  describe  how  to  relate  the  complexity  of 
networks  to  generalization  performance  on  unseen  test  data.  Many  new  network  architec¬ 
tures  were  described.  Some  integrate  expen  system  rules  with  networks,  build  hierarchies  of 
networks,  use  radial  basis  funaion  hidden  nodes,  and  impose  pre-specified  invariance  on  the 
final  solution.  Neurobiological  papers  analyzed  and  modeled  neurons  in  the  hippocampus,  in 
cat  striate  conex,  and  in  the  blowfly.  They  also  modeled  biological  networks  that  control  eye 
movement,  form  topological  maps,  and  compensate  for  head  movement.  Successful  applica¬ 
tions  of  neural  networks  tvere  described  in  the  areas  of  speech,  vision,  language,  control, 
medical  monitoring,  and  system  diagnostics.  Of  particular  interest  was  a  paper  by  Tesauro, 
which  demonstrated  how  a  network  could  be  trained  to  play  backgammon  at  an  expen  level; 
papers  by  Jain,  Watrous,  and  Giles,  which  described  approaches  to  learning  grammars; 
hybrid  hidden-Markov-model/neural-network  speech  recognizers  described  by  Haffner, 
Levin,  Singer,  Renals.  and  Bengio;  papers  on  optical  charaacr  recognition;  a  paper  by  jabri, 
which  describes  a  network  to  control  a  wearable  hean  defibrillator;  a  paper  by  Smyth  for 
diagnosis  of  large-dish  antenna  poindng  systems;  and  a  paper  by  Rdscheisen  concerning 
control  of  force  on  rollers  in  steel  rolling  mills.  Papers  also  described  new  analog  and  digital 
VLSI  chip,  systems  for  neural  network  implementation,  and  compared  neural  network  and 
sutistical  approaches  to  pattern  classification. 
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An  histoncai  milestone  was  reached  this  year,  NIPS-91  was  the  fifth  NIPS  conference 
since  the  first  conference  was  hdd  in  1987.  To  mark  this  anniversary,  we  dedded  to  review 
the  history  of  events  that  led  to  the  foundation  of  the  NIPS  conference  artd  to  discuss  the 
evolution  of  the  conference  since  iu  foundation.  The  following  history  is  based  in  part  on 
the  recoilcctioru  of  Jim  Bower.  Larry  jackd,  and  Ed  Posner.  Some  of  thu  history  was  pre¬ 
sented  by  Larry  Jacket  at  the  opening  banquet. 

While  the  fint  NIPS  conference  met  in  1987,  io  origiiu  can  be  traced  back  to  the 
“Hopfcst’  meetings  turned  in  honor  of  John  Hopsfield,  held  at  Caltech.  The  first  few, 
1984-1986,  were  organized  by  Ed  Posner  of  Caltech.  These  meetings  noet  in  the  foil  and 
included  researches  mainly  from  the  Caltech  campus  and  JPL  In  1985.  Larry  jackd  of  Bell 
Labs  and  Demetri  Psaltis  of  Caltech  organized  the  first  of  what  were  to  become  the  ‘Snow¬ 
bird’  meetirtgs.  The  meetings  were  intended  to  be  small  informal  workshops  and  convened 
in  Sanu  Barbara.  Twenty  people  were  invited,  but  news  of  the  meeting  spread  by  word  of 
mouth,  so  that  anendance  ended  up  growing  to  60.  In  1986,  the  meetittg  reconvened  at 
Snowbird,  which  offered  bener  snow  condidons.  Jackel,  Psalds,  and  the  other  organizen 
intended  to  keep  the  attendance  down  to  100  people,  but  the  interest  was  so  great  that  rtuny 
people  were  turned  away  even  after  the  attendance  was  capped  at  160.  The  first  Snowbird 
proceedings  was  edited  by  John  Oenker  of  Bell  Labs  and  published  by  the  American  Insdtute 
of  Physics  (AIP)  press. 

In  1986,  the  Snowbird  meedng  was  the  only  neural  network  confererKC,  artd  it  dearly 
could  not  accommodate  the  exploding  numbers  of  researchers  becoming  interested  in  the 
field  and  sail  iruintain  the  character  of  a  small  workshop.  To  respond  to  demartd,  the 
organizers  decided  to  make  Snowbird  a  more  dosed  meeting,  but  to  set  in  motion  organiza¬ 
tion  of  a  large  meeting  that  would  be  open  to  all  interested.  The  goal  was  to  have  a  norv 
commercial  meeting  dedicated  to  scholarship,  which  would  capture  some  of  the  flavor  of  the 
workshop.  The  Snowbird  organizen  nomiiuted  a  committee  with  Ed  Posner  as  General 
Chairman  and  Yaser  Abu  Mostafa  as  Program  Chairman  (both  of  Caltech),  to  organize  and 
run  the  1987  NIPS  conferetKe,  which  was  officially  sponsored  by  the  IEEE  Information 
Theory  Sodety.  Denver  was  chosen  as  the  site  due  to  its  central  geographical  location,  ease  of 
access  by  air,  and  dose  proximity  to  the  mountains  and  the  Univenity  of  Colorado  at 
Boulder. 

The  1987  organizen  designed  the  NIPS  conference  to  have  many  of  the  advanta^  of  a 
workshop,  while  sail  accommodating  a  large  audience.  To  maximize  scientific  interchange, 
they  dedded  to  limit  the  oral  presentations  to  a  single  stream,  have  posten  be  the  majority  of 
presentatioru,  and  tndude  poster  preview  as  well  as  formal  poster  sessioru.  Furthermore,  a  set 
of  post-conference  workshops  was  organized  at  the  Copper  Mountain  ski  resort  after  the 
main  conference  to  eruble  small  groups  to  discuss  spedfic  topics.  The  1987  conferetKe 
proved  to  be  a  great  success,  with  about  450  attendees  and  91  papen  making  it  into  the 
proceedings.  Dana  Z.  Anderson  of  CU  Boulder  edited  the  proceedings,  which  were  pub¬ 
lished  by  the  AIP  press  and  are  now  informally  known  as  NIPS  Volume  0. 

Since  1987,  some  changes  and  refinements  have  been  made,  but  the  basic  structure  of 
the  conference  has  remained  the  same.  The  NIPS  1988  proceedings  (NIPS  Volume  1,  edited 
by  David  Tourcstzky  of  Carnegie  Mellon)  were  the  first  published  by  Morgan  Kaufmann. 
Also  in  1988,  the  post-conference  workshops  were  moved  to  Keystone,  CO.  The  refinement 
processes  (three  reviewere  instead  of  two),  a  more  cross-disdplinary  grouping  of  presena- 
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dons,  finer  presenution  catagohes,  and  the  addition  of  five-minute  oral  potter  spotlight 
presentations.  A  major  and  very  successful  addition  to  the  1991  conference  was  the  introduc¬ 
tion  of  a  day  of  tutorials  preceding  the  main  conference.  The  1991  woriuhops  were  held  at 
Vail,  which  proved  to  ba  a  popular  move. 

Finally,  1991  marked  the  drafting  of  articles  of  incorporation  for  the  Neural  Irtforma- 
tion  Processing  Systems  Foundation,  which  will  be  respoiuible  for  the  continuiry  of  the 
NIPS  conference  in  future  years.  The  initial  board  of  diteaors  of  the  foundation  consists  of 
the  1987  to  1992  NIPS  General  Chairs  (Ed  Posner  of  Cal  Tech.  Terry  Sejnowski  of  the  Salk 
Insdtute  and  (J<3SD.  Soon  Kirkpatrick  of  IBM.  Richard  Lippmann  of  MIT  Lincoln  Labs. 
John  Moody  of  Yale,  and  Stephen  Hanson  of  Siemens),  a  member  of  the  IEEE  litformation 
Theory  Society  (Terry  Fine  of  Cornell),  and  our  leg^  counsel  (Philip  Sotel). 

The  NIPS  conference  condnues  to  be  an  exciting,  successful  meedng  due  to  the  effons 
of  a  large  group  of  people.  We  would  first  like  to  thank  all  the  other  members  of  the  1991 
program  and  organizing  committees  who  helped  make  this  conference  possible.  In  particular, 
we  would  like  to  thank  Rerute  Crowley  of  Siemens  for  her  extensive  work  throughour  the 
year  as  the  conference  secretary  and  both  Renate  and  Kate  Fuqua  of  CU  Boulder  for  running 
the  confereiKe  desk  so  smoothly.  Student  contributions  are  an  important  pan  of  the  NIPS 
program,  and  we  gratefully  thank  Tom  McKenru  of  ONR  and  Steve  Suddanh  of  AFOSR 
for  the  student  travel  funding  provided  by  their  agencies.  Finally,  we  thank  everyone  who 
attended  and  submitted  papers  and  the  105  referees  who  carefully  read  and  reviewed  20 
papers  each. 

John  Moo^y 
Sttphen  Hanson 
Richard  Lippmann 


xvit 


NIPS-91  ORGANIZING  COMMITTEE 


General  Chair 
Program  Chair 
Workshop  CoChair 

Publicity  Chair 
Publications  Chair 
Treasurer 

Governmeni/Corporate  Liaison 
Local  Arrangements  Chair 
IEEE  Liaisons 
Tutorials  Chair 
APS  Liaisons 

Neurobiology  Liaisons 

IEEE  Liaisons 
Overseas  Liaison  (japan) 

Overseas  Liaison  (Australia) 
Overseas  Liaison  (United  Kingdom) 
Overseas  Liaison  (South  America) 


John  Moody.  Yale  University 

Stephen  Hanson,  Siemens 

Gerry  Tesauro,  IBM 

Scott  Kirkpatrick,  IBM 

John  Pearson,  David  SarnolT  Research  Lab 

Richard  Lippmann,  MIT  Lincoln  Laboratory 

Bob  Allen,  ^llcore 

Lee  Giles,  NEC  Research  Institute 

Mike  Mozer,  University  of  Colorado 

Rodney  Goodman  and  Ed  Posner,  Caltech 

John  Moody,  Yale  University 

Eric  Baum,  NEC  Research  Institute 

Larry  Jacket.  AT&T  Bell  Labs 

James  Bower,  Caltech 

Tom  Brown,  Yale  University 

Rodney  Goodman  and  Ed  Posner,  Caltech 

Mitsuo  Kawato.  ATR  Research  Laboratories 

Maran  Jabti,  University  of  Sydney 

John  Bridle.  RSRE 

Andreas  Meier,  Simon  Bolivar  University 


NIPS-91  PROGRAM  COMMITTEE 


Program  Chair  Stephen  Hanson,  Siemens 

Program  CoChain  David  Ackley,  Bellcore 

Pierre  Baldi.  JPL  and  Calteh 
William  Bialek,  NEC  Research  Itudtute 
Lee  Giles.  NEC  Research  Institute 
Steve  Omohundro,  ICSI 
Mike  Jordan,  MIT 
John  Platt,  Synaptics 
Terry  Sejnowski,  Salk  Institute 
David  Stork,  Ricoh  and  Stanford 
Alex  Waibel.  CMU 


zviii 


NIPS-91  Rcview'ers 


Manin  Brady,  I^nn.  Suk  Uttiv. 

Gary  Cortrell,  UCSD 
John  Denker,  AT&T  Lab 
John  Hero,  NORDITA 
James  Keeler,  Advanced  G>mputing  Tech¬ 
nology 

Alan  Lapedes,  Los  Alamos 
David  Rogers,  RIACS 
David  G.  Stork,  Ricoh 
Peter  Todd.  Stanford  Univ. 

Andreas  Weigend,  Stanford  Univ. 

Sue  Becker,  Univ.  of  Toronto 
David  A.  Cohn,  Univ.  of  Washington 
Michael  Littman,  Bellcore 
David  Plaut,  CMU 
Lori  Pratt,  Rutgers  Univ. 

Jiirgen  Schmidhuber,  Technische  Univ. 
Muenchen 

Rich  Zemel,  Univ.  of  Toronto 
Paul  Munro,  Siemens 
Ben  Yuhas,  Bellcore 
Steve  Gallant,  Nonheastern  Univ. 

Marwen  Jabri,  Sydney,  Univ 
Gary  M.  Kuhn,  CCRP-IDA 
Y.C.  Lee,  Univ,  of  Maryland 
Ken  Marko,  Ford  Motor  Company 
Michiel  Noordewier,  Rutgers  Univ. 

Kevin  Lang,  NEC 
Steven  C.  Suddarth,  AFOSR 
Robert  Alien,  Bellcore 
Jonathan  Bachrach 
Gary  Dell,  Beckman  Institute 
Michael  G.  Dyer.  UCLA 
Jeff  Elman.  UCSD 
Michael  Gasser,  UCSD 
Lee  Giles.  NEC 
Robert  Jacobs,  MIT 
Stephen  Hanson,  Siemens 
Christopher  Atkeson,  MIT 


Andrew  Bano,  Univ.  of  Missachusetis 
Judy  Franklin,  GTE 

Vijaykumar  Gullipalli,  Univ.  ofMasuchu- 
setts 

Thomas  Maninez.  Beckman  Institute 
Kumpati  Narendra,  Yale  Univ. 

Richard  Sunon,  GTE 
ManocI  Tenorio,  Purdue  Univ. 

Lyle  Ungar,  Univ.  of  Peimsylvania 
Josh  Alspeaor,  Bellcore 
Jim  Burr.  Stanford  Univ. 

Federico  Faggin,  Synaptics 
Hans  Peter  Graf,  AT&T 
Kristina  Johnson,  Univ.  of  Colorado 
John  Lazzaro,  Univ.  of  Colorado 
Dick  Lyon.  Apple  Computer 
Dan  Schwartz,  GTE 
Larry  Abbon,  Brandeis  Univ. 

Thomas  Anastasio,  Univ.  of  Southern 
California 

Joseph  Attick,  Princeton 
William  Biaiek,  Univ.  of  California  at 
Berkeley 

A.B.  Bonds,  Vanderbilt  Univ. 

James  Bower,  California  Inst,  of  Tech. 
Thomas  Brown,  Yale  Univ 
Jack  Cowan,  Univ.  of  Chicago 
Shawn  Lockery,  Salk  Institute 
Bartlett  Mel,  California  Inst,  of  Tech. 
Kenneth  Miller,  California  Inst,  of  Tech. 
Eric  Schwartz,  New  York  Univ.  Medical 
Center 

Terry  Sejnowski,  Salk  Institute 
Gordon  Shepherd,  New  Haven 
David  van  Essen,  C^fomia  Inst,  of  Tech. 
Frank  Falbide.  Univ.  of  Cambridge 
Herv<  Bourlard,  Phillips 
Patrick  Haffher,  Centre  National 
d'Etudes  des  Telecommunications 


xtx 


Riclurd  Lippmann,  MIT  Lincoln  Lab 
Hong  Leung,  MIT 
John  S.  Bridle,  RSRE 
David  Burr.  Bellcore 
Richard  Durbin,  ICSI 
David  Hauauler,  Univ.  of  California  at 
Santa  Cruz 
Terry  Sanger,  MIT 
Gerry  Tesauro,  IBM  Watson  Lab 
Eric  Mioisness,  Yale  University 
Ronny  Meir,  Bellcore 
Sara  Solla,  Sanu  Fe  Institute 
Fernando  Pineda,  Applied  Physics  Lab 
Steve  Oinohundro,  ICSI 


Pierre  Baidi,  California  Inst,  of  Tech. 
Amir  Aliya.  Texas  A&M  Univ. 

Santosh  S.  Venkatesh,  Uruv.  of  Pennsyl¬ 
vania 

Halben  White,  UCSD 
Stephen  Judd,  Siemens 
Tom  Petsche.  Siemens 
Subatai  Ahmad.  ICSI 
Joachim  Buhmann,  Univ.  of  Southern 
California 

Gene  Gindi.  Yale  Univ. 

Geoff  Hinton,  Univ.  of  Toronto 
Nathan  Intracor,  Brown  Univ. 


